The growing use of smartphones and other hand-held computing devices, equipped with high-resolution cameras and large-view touchscreens, has significantly increased multimedia contents on the Internet. This increase includes the amount of multimedia content available to users nowadays, as well as the amount of multimedia data consumed or produced (e.g., searched for, viewed, streamed, or uploaded) by them and transmitted over the Internet. The wide availability of complimentary or low-cost wireless Internet access (Wi-Fi) and increased use of more affordable mobile data plans also contribute to such increase.
This presents several challenges for traditional distributed data processing systems to process the multimedia contents. First, most existing systems and algorithms are not specifically designed or optimized to process photos, audio, or video contents. For example, unlike indexing a textual webpage where its keywords are typically available within that webpage, the task of “understanding” and indexing a photo with no additional description is much more difficult, often requiring complex machine learning and/or image analysis algorithms. Moreover, running those algorithms can be much more demanding on computation resources. Secondly, a multimedia content file can be larger than a textual counterpart by many orders of magnitude. For example, a novel in a textual format can be well within hundreds of kilobytes (KBs), whereas a High-Definition movie based on that novel can easily take several gigabytes (GBs), depending on the movie file resolution and its compression method and/or level. This difference can necessitate re-designing existing systems to process the multimedia contents efficiently. Also, to allow different applications to work on the same distributed processing system without significant or frequent changes to the system, it is desirable to provide applications with some flexibility to use such a system.